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Surprise is a central concept in learning, attention and the 
study of the neural basis of behaviour. However, how sur- 
prise affects learning and more specifically, how surprise 
affects synaptic learning rules in neural networks is largely 
undetermined. Here we study how surprise facilitates 



learning in different environments and how surprise can 
potentially modulate Hebbian learning in the form of a 
global factor in multi-factor learning rules. 

Learning rate is a crucial factor in determining to 
what extent the learning agent should rely on the newly 
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Figure 1 Estimation of the probability of reward delivery in a reversal task. A. Estimated reward rate. B. Reward prediction error. C. Surprise 
measure. D. Learning rate. E. Uncertainty measure. F. Optimal Kalman learner. 
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acquired information rather than the old information in 
building its own internal model of the external world. 
Both theory and empirical evidences suggest that the 
learning rate should be adjusted under different circum- 
stances for having an optimal and effective learning 
strategy. We propose a simple and biologically plausible 
model that describes the dynamics of the learning rate 
in terms of surprise and uncertainty measures. We 
apply our model to three different tasks: a reversal task 
(Fig. 1), a dynamic decision making task, and a dynamic 
clustering task. 

Our proposed model explains how the agent should 
effectively control the speed of learning in different envir- 
onments such that it matches both theory and empirical 
evidences from human and animal subjects. This model 
explains why surprising events provoke humans and ani- 
mals to learn faster and why they rapidly adapt to chan- 
ging environments. It also addresses the question of what 
the effective learning rate should be in both stable (either 
low-risky or high-risky) and volatile environments. Here 
effectiveness is defined as having a higher accuracy in 
learning a task, for instance the estimation of the mean 
reward in classic reinforcement learning, for a given time 
and computational complexity as well as the available 
memory as our constraints. This study also suggests a 
functional connectivity pattern for the neurochemical sys- 
tems that are related to contextual modulation of learning 
rate. Further, it explains why we need different neuromo- 
dulators with distinct functional roles to act in parallel in a 
broad range of distribution and proposes suitable candi- 
dates responsible for measuring different quantities we 
need in the model. 
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